As compression coding systems for dynamic image signals, MPEG-2 (ISO/IEC 13818-2), MPEG-4 (ISO/IEC 14496-2) and the like are known. In MPEG-2 and MPEG-4, a dynamic image signal is compressed by a coding system utilizing motion prediction. In MPEG-2 and MPEG-4, motion prediction between frames is performed by block, with each block including, for example, 16×16 or 16×8 pixels. In MPEG-2 and MPEG-4, the predicted quantity of motion is expressed by a vector (called motion vector) and it is included in coded data. In MPEG-2 and MPEG-4, at the time of coding, a prediction frame is generated from a previously coded frame by utilizing a motion vector. Then, a difference between a coding target frame and the prediction frame is calculated and only that difference is coded, for example, by DCT (discrete cosine transform). In MPEG-2 and MPEG-4, at the time of decoding, a prediction frame is generated by using a motion vector with respect to a previously decoded reference frame, and the prediction frame and the decoded frame are added to restore an original image.
In the coding system utilizing motion prediction such as MPEG-2 and MPEG-4, since it suffices to code only a motion vector and a differential image, dynamic image data can be compressed efficiently.
Next, a conventional MPEG decoding apparatus for decoding a dynamic image data stream (elementary stream) compression-coded by the MPEG-2 system or the MPEG-4 system will be described.
FIG. 1 shows a block structural diagram of a conventional MPEG decoding apparatus.
A conventional MPEG decoding apparatus 100 has a syntax analyzer circuit 111, a variable-length code (VLC) table 112, an inverse quantizer (IQ) circuit 113, an inverse discrete cosine transform (IDCT) circuit 114, a motion compensation circuit 115, a frame memory 116, and a color format converter circuit 117, as shown in FIG. 1. Inside the motion compensation circuit 115, a selector 121, an MC controller 122 and an adder 123 are provided.
An elementary stream is inputted to the syntax analyzer circuit 111. The syntax analyzer circuit 111 detects a fixed-length code, analyzes a code length of a variable-length code and calculates an address of a variable-length code table, from the inputted elementary stream. The syntax analyzer circuit 111 detects a code that is before variable-length coding, from the calculated address with reference to the VLC table 112. The syntax analyzer circuit 111 detects these fixed-length code and variable-length code, thereby decoding DCT coefficient data, motion vector, and various control data for inverse quantization and motion compensation. The DCT coefficient data is rearranged in a predetermined scanning order and then supplied to the inverse quantizer circuit 113. The motion vector and the control data for motion compensation are supplied to the MC controller 122 of the motion compensation circuit 115.
The syntax analyzer circuit 111 outputs an error flag when a code that does not exist in the VLC table 112 is inputted to the inputted elementary stream, or when an error occurs in DCT coefficients to be outputted, for example, when the number of DCT coefficients in one DCT block exceeds 64. The error flag is generated by macroblock. The error flag is supplied to the MC controller 122 in the motion compensation circuit 115.
The inverse quantizer circuit 113 performs inverse quantization of the DCT coefficient data, using a quantization parameter used in coding. The inversely quantized DCT coefficient data is supplied to the inverse discrete cosine transform circuit 114.
The inverse discrete cosine transform circuit 114 performs two-dimensional inverse discrete cosine transform operation of each DCT block including 8×8 DCT coefficients, of the DCT coefficient data, and decodes image data of the spatial domain. The image data transformed by inverse discrete cosine transform is supplied to the motion compensation circuit 115.
The motion compensation circuit 115 performs motion compensation processing of the image data of the spatial domain that has been transformed by inverse discrete cosine transform.
The MC controller 122 in the motion compensation circuit 115 performs motion prediction by macroblock with respect to a reference frame stored in the frame memory 116 on the basis of the motion vector and the control data for motion compensation supplied from the syntax analyzer circuit 111, and thus generates a prediction frame. The adder 123 in the motion compensation circuit 115 adds, by macroblock, the prediction frame generated from the reference frame to the differential frame supplied from the inverse discrete cosine transform circuit 114, and thus generates an original frame. The frame outputted from the adder 123 is stored into the frame memory 116 as a decoded frame.
The selector 121 in the motion compensation circuit 115 performs processing to stop, by macroblock, the output of the differential frame from the inverse discrete cosine transform circuit 114 at the time of error conceal processing.
The error conceal processing is the processing to reduce visual imperfection in an error part, for example, if an error occurs in coded data at the time of transmission and there is a macroblock that is extremely different from the peripheral normal image.
The MC controller 122 performs the error conceal processing of a macroblock at which an error flag is raised by the syntax analyzer circuit 111. When the macroblock at which an error flag is raised by the syntax analyzer circuit 111, that is, a macroblock in which an error has occurred and its image has been broken, is a target macroblock of motion compensation, the MC controller 122 does not perform ordinary motion compensation but performs processing to interpolate the pixel having the error in the macroblock with a pixel of the reference frame that is at the same position as that macroblock, thus performing the error conceal processing. Specifically, the MC controller 122 uses a mode for referring to I-picture or P-picture, as an MB mode, which is a mode for designating a reference frame of that macroblock, and sets the motion vector value at 0, which indicates no motion from the reference frame. Then, the MC controller 122 sets the value of the macroblock as a decoding target at 0. That is, the MC controller 122 performs the error conceal processing by switching the selector 121 to set 0 as the input data from the inverse discrete cosine transform circuit 114 to the motion compensation circuit 115.
The color format converter circuit 117 performs video format conversion processing of the decoded image data stored in the frame memory 116. For example, the color format converter circuit 117 performs format conversion to the 4:2:2 format or 4:2:0 format prescribed by ITU-R601.
As described above, in the conventional MPEG decoding apparatus 100, coded data utilizing motion prediction can be decoded. In the conventional MPEG decoding apparatus 100, even if an image block broken at the time of transmission exists in a frame, the error conceal processing can be performed to interpolate the image block with an image block of another frame that is not broken. Therefore, it is possible to reduce visual imperfection due to the presence of a block that is extremely different from the peripheral normal image because of an error.
Meanwhile, in the case of performing motion compensation, if an error-concealed frame becomes a reference frame and a pixel within an error-concealed macroblock is referred to for motion compensation, motion compensation is performed on the basis of an erroneous reference value. If motion compensation is performed on the basis of such an erroneous reference value, the image within that macroblock is broken as a picture. In the case of performing motion compensation of the next frame, a pixel within the macroblock that is broken as a picture might be further referred to. That is, if the error conceal processing is performed, the error might be propagated temporally and spatially.
The temporal and spatial propagation of the influence of an error-concealed macroblock will now be described with reference to FIGS. 2, 3 and 4.
First, as shown in FIG. 2, it is assumed that a transmission error occurs in two macroblocks (MB3, MB4) in a certain I-picture (I1). In this case, as the error conceal processing is performed, these macroblocks (MB3, MB4) are interpolated, for example, with macroblocks (MB3, MB4) at the same positions in a P-picture (P0) that is immediately before I1. Therefore, the image of the macroblocks (MB3, MB4) of decoded I1 is different from the original picture but is not broken as a picture.
The next P-picture (P2) uses I1 as a reference frame for motion compensation. Since no transmission errors occur in this P2, the error conceal processing is performed. However, for example, as shown in FIG. 3, three macroblocks (MB1, MB3, MB4) in P2 contain the pixels of the error-concealed macroblocks in the reference frame, in a reference area for motion compensation. Therefore, the image of the decoded macroblocks (MB1, MB3, MB4) is different from the original picture and its picture is broken.
Subsequently, the next P-picture (P3) uses P2 as a reference frame for motion compensation, as shown in FIG. 4. A transmission error occurs in a certain macroblock (MB4) of this P3. In this case, as the error conceal processing is performed, the macroblock (MB4) is interpolated, for example, with the macroblock (MB4) at the same position in the P-picture (P2) that is immediately before P3. However, since the previous macroblock (MB4) as the interpolation source has its picture already broken, the image of the macroblock (MB4) to be interpolated differs from the original picture and its picture is broken.
Moreover, P-picture (P3) contains the pixels of the error-concealed macroblock in the reference frame, for example, in a reference area for motion compensation of certain macroblocks (MB1, MB2, MB3). Therefore, the image of the decoded macroblocks (MB1, MB2, MB3) differs from the original picture and its picture is broken.
As described above, in the case the error conceal processing is performed, the error is propagated until the next I-picture (I4) is decoded, and the error is spread in a planar direction by motion compensation. Such expansion of error seriously deteriorates the image.